1. Field of the Invention
The present invention relates to color correction of images, and in particular relates to methods and apparatus for color device characterization, correcting color of a color image obtained by an electronic camera, transmission of color corrected data and a recording medium having recorded thereon a color corrected data of a color image.
2. Description of the Related Art
There are known in the art two approaches for performing color correction for an image captured by an electronic color camera. A first approach is to directly convert the camera's output data into the same color space as camera's output color space with improved color accuracy. For instance, when the camera's output space is YUV (defined below), the color correction model's input and output spaces are YUV, but with higher color accuracy at the output YUV. Hereinafter, this approach will be referred to as “device specific color correction.” “YUV” is a color representation which is typically used in connection with video transmission. “Y” represents luminance, “U” represents a component of colors corresponding approximately to blue minus yellow, while “V” represents component of colors corresponding approximately to red minus cyan. Other color representations include “RGB” (red, green blue) and “YCC” where YCC indicates any color space consisting of one luminance factor and two chrominance factors. The above-described YUV is one example of a YCC color representation. YIQ, is another example of a YCC color representation, where “I” is a component of colors corresponding approximately to yellowish red minus light blue, while “Q” is a component of colors corresponding approximately to violet minus green-ish yellow. YCbCr (luminance, chrominance-blue, chrominance-red is yet another example of a YCC color representation.
A second approach for color correction is divided into two steps. First, the camera's output color space is once converted into a calorimetric space (XYZ or L*a*b*), the space which is specified by CIE (International Illumination Committee, originally published in the French language). A calorimetric color space attempts to describe color of natural objects perceived by human eyes. Color data described in a calorimetric space corresponds to the original color of the images taken by an electronic color camera. In a second step, the color data converted into such a colorimetric space is then converted into an output device's (computer monitor or printer) specific color space, where the entire image path can provide calorimetrically correct color reproduction. This two-step concept is incorporated in the ICC (International Color Committee) color profile format, which is commonly used in computer image processing where relatively high accuracy is required in color reproduction. In the context of the color correction based on ICC's concept, the intermediate calorimetric space is often referred to as a “device independent color space.” Therefore, this color correction approach is referred to as “device independent color correction.”
To perform the above-described device-independent color correction, one should attempt to determine an inverse function of a camera's color transformation. This step is called “color device characterization” for an electronic camera, and such characterization is employed to correct color of images obtained by the electronic camera. The conventional practice for determining the inverse function of an electronic camera's color transformation is to apply a method widely used for determining an inverse function of image scanners' color transformation. This approach has been considered reasonable, because most color cameras have signal processing similar to that of image scanners.
FIG. 1 shows the signal-processing diagram that is used in both image scanners and most electronic color cameras. In FIG. 1, input data of image sensor 104 of camera 100 is assumed to be in calorimetric space XYZ, which is provided to inputs 101, 102 and 103, respectively. The output space of image sensor 104 is RGB, while the camera's output space is YCC. Camera output is provided at outputs 110, 111, and 112, corresponding to Y, C and C, respectively.
A simple reverse-model, shown in FIG. 2, can be used to approximate the inverse function of the color transformation formed by the image sensor camera 100 shown in FIG. 1. In FIG. 2, each block should perform inverse conversion of corresponding blocks in FIG. 1. The pairs of corresponding blocks are 104–124, 151–161, 152–162, 153–163 and 109–122. Specifically, matrices 104 and 109 should be inverse matrices of 124 and 122 respectively. Tone reproduction curves (“TRCs”), which are usually implemented by one-dimensional lookup tables (“1D-LUTs”) 151, 152, and 153, should be inverse functions of 1D-LUTs 161, 162 and 163, respectively. Since this color correction model consists of matrices and TRCs, it is called a “TRC-matrix model.”
As shown in FIGS. 1 and 2, the color transformation of image sensor 104 is linearly approximated by a 3×3 matrix 124. According to color science for electronic cameras, an image sensor's spectral analysis characteristics ideally should be a linear combinations of color matching functions xyz specified by CIE, that is, a set of spectral functions which represent human eye's color sensitivity. Since all color cameras are designed to target these ideal characteristics, in many cases a 3×3 matrix can well approximate an image sensor used in electronic cameras, thereby providing a reasonably accurate inverse function of the camera's color transformation. This TRC-matrix model approach, however, cannot perform an accurate color correction under some circumstances.
Approximation Accuracy
When an image sensor's color transformation cannot be well approximated by a 3×3 matrix in the TRC-matrix model (124 in FIG. 2), color correction cannot be accurate. This situation can likely occur in many entry-level electronic cameras. This is also the case when the camera's signal processing does not match the configuration shown in FIG. 1, such that the configuration of FIG. 2 can no longer be a reverse model of the camera. For instance, FIG. 3 is a block diagram which illustrates signal processing typically used in a CMYG (Cyan, Magenta, Yellow, and Green, also referred to as “complementary color”) electronic color camera. At first glance, FIG. 2 cannot work as a reverse model for the signal processing of FIG. 3. In addition, it is not easy to construct a simple reverse model for the signal processing performed by the system of FIG. 3, due to structural complexity of a CMYG camera's signal processing.
Illuminant Dependency
In the case of an electronic camera, whether it be a still camera or one intended to record a succession of still or moving images, there is a somewhat unique problem, more specifically, that of illuminant dependency of color transformation. While an image scanner captures images using one fixed built-in illuminant all the time, an electronic camera captures images under a variety of illuminants, for example, outdoors at dawn, mid-day or late afternoon, indoors with fluorescent, incandescent or other types of artificial lighting, a combination of artificial lighting, or even a combination of natural (outdoor) and artificial lighting. In an electronic camera, there is a mechanism to estimate the illuminant at the time of image capturing, wherein the camera's gain settings at R, G, and B channels are adjusted according to the on-camera illuminant estimation, so that the final output image has R=G=B with respect to a white object. This operation is called “white balancing,” and the gain adjustment is placed between image sensor 104 and 1D-LUTs 105 in FIG. 1, although it is not shown in the diagram. Usually, color transformation of a camera changes depending on the setting of white balancing. In other words, an electronic camera has illuminant dependency in its color transformation. It means color distortion or error caused by an electronic camera changes depending on illuminant. Therefore, for highly accurate color correction, it is necessary to prepare for inverse functions of the camera's color transformation under all possible illuminants, which is impractical. One existing solution for this problem is to prepare for a set of inverse functions (likely in a form of ICC color profile, using either a TRC-matrix model or a three-dimensional lookup table) of color transformation under several representative illuminants. A user then has to manually select from the set of preset inverse functions, considering which inverse function is the best fit to the illumination under which the user actually captured the image. This not only is a burden for the user, but also can result in inaccurate color correction if the user's selection is inappropriate. Furthermore, inaccurate correction can also occur when the actual illumination used for the image capture falls between the preset inverse functions. This is a limitation of the lookup-table based (TRC-matrix) method. Another limitation of the look-up table based method is its inability to accurately correct color under multi-illuminant conditions. An alternative to the TRC-matrix method is the three dimensional lookup table (“3D-LUT”). Although such a 3D-LUT is more accurate than the TRC-matrix, the size of the correction model is huge, and it is, as with the TRC-matrix method, incapable of accurately correcting color under multi-illuminant conditions. Thus, it would be desirable to provide a method and apparatus for accurately correcting the color of images taken by an electronic camera under arbitrary illuminants.